1. Field of the Invention
The present invention relates to an image processing device, an imaging device, an image processing method, and non-transitory computer readable recording medium storing an image processing program regarding restoration processing based on a point spread function.
2. Description of the Related Art
In an object image captured through an optical system, a point spread phenomenon in which a point object has minute spread due to the influence of diffraction, aberration, or the like caused by the optical system may be observed. A function representing a response to a point light source of an optical system is called a point spread function (PSF), and is known as a characteristic responsible for resolution deterioration (blurring) of a captured image.
The image quality of the captured image with deteriorated image quality due to the point spread phenomenon can be recovered through point image restoration processing based on the PSF. The point image restoration processing is processing in which deterioration characteristics (point image characteristics) caused by aberration or the like of a lens (optical system) are determined in advance, and point spread of the captured image is cancelled or reduced through image processing using a restoration filter (recovery filter) according to the point image characteristics.
The point image restoration processing can be roughly classified into frequency recovery (correction) processing and phase recovery (correction) processing. The frequency recovery processing equalizes, that is, recovers modulation transfer function (MTF) characteristics deteriorated by an optical system, and the phase recovery processing equalizes, that is, recovers phase transfer function (PTF) characteristics deteriorated by an optical system.
Intuitively, the phase recovery processing moves an image in a frequency-dependent manner so as to return an asymmetrical PSF shape to as symmetrical a shape as possible.
While the frequency recovery processing and the phase recovery processing can be applied simultaneously as signal processing, either correction may be performed by changing a design method of a filter coefficient.
Various methods for this kind of point image restoration processing have been suggested.
JP2011-124692A discloses an image processing device which, in order to suppress amplification of noise due to image recovery processing, or the like, performs recovery processing on an amplitude (frequency) component and a phase component of an input image to generate a first image, performs recovery processing only on the phase component of the input image to generate a second image, acquires difference information between the generated first and second images, and synthesizes the difference information with the second image according to an appropriate recovery intensity adjustment coefficient to generate a recovery adjusted image.
JP2011-059813A discloses an image processing device which performs image recovery for an image subjected to nonlinear correction using blind deconvolution. This image processing device comprises a correction unit which performs correction for reducing nonlinear gradation correction for a captured image subjected to nonlinear gradation correction, and an image recovery unit which performs image recovery by applying blind deconvolution to the captured image with reduced gradation correction.
JP2013-020610A discloses an image processing device which reduces over-recovery of image data by image recovery processing. In this image processing device, image recovery processing is performed for color image data in an RGB format before gamma processing, the difference between amplification and attenuation of a pixel signal value by gamma correction is absorbed, and a limit value of a variation is calculated such that the maximum value of the variation of the pixel signal value becomes constant even after gamma correction. With this, the technical problems in that “a situation in which a deterioration state of image data actually obtained does not match a deterioration state of image data to be recovered by an image recovery filter occurs due to a saturated pixel”, and “image quality deterioration, such as undershoot or overshoot, occurs in an edge portion, and in particular, undershoot in a low brightness portion is amplified by gamma processing after image recovery processing” have been solved.
The point spread function of the optical system is used in a restoration technique of an image with an extended depth of focus, and JP2012-049759A discloses an imaging module which executes image restoration in a short period of time with excellent accuracy. In this imaging module, restoration processing is applied to a brightness signal after synchronization processing (demosaic processing), whereby it is not necessary to separately provide parameters of the restoration processing for RGB, and it is possible to accelerate the restoration processing. Furthermore, adjacent pixels are put together in a predetermined unit and the common restoration processing parameters are applied to this unit to perform a deconvolution processing, whereby the accuracy of the restoration processing is improved.